Method to predict food color and recommend changes to achieve a target food color

ABSTRACT

A color predictor is provided to predict the color of a food item given its formula comprising the ingredients and its quantities. The color predictor may utilize machine learning algorithms and a set of recipe data to train the color predictor. The color predictor can also be used by a color recommender to recommend changes in the given formula to achieve a target color.

BENEFIT CLAIM

This application is a continuation of U.S. non-provisional applicationSer. No. 16/596,689, filed Oct. 8, 2019, the entire contents of which ishereby incorporated by reference for all purposes as if fully set forthherein. The applicant(s) hereby rescind any disclaimer of claim scope inthe parent application(s) or the prosecution history thereof and advisethe USPTO that the claims in this application may be broader than anyclaim in the parent application(s).

TECHNICAL FIELD

The disclosure generally relates to food science and artificialintelligence, in particular, use of machine learning to predict color ofa food item and/or recommend changes to achieve a target color.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates a system comprising a color predictor configured topredict the color of a food item given its formula in certainembodiments;

FIG. 2 illustrates a block diagram of the color predictor in certainembodiments;

FIG. 3 illustrates an example encoding vector generated using themulti-warm encoder for an example given formula according to certainembodiments;

FIG. 4 illustrates an embodiment of a system which can utilize aningredient embedder to incorporate unsupervised data for training anembedding layer, in certain embodiments;

FIG. 5 illustrates an embodiment of a system comprising a colorrecommender configured to provide a recommendation for changes in thegiven formula for a food item to achieve the desired color of the fooditem;

FIG. 6 illustrates a block diagram that can be used to describe the lossfunction model implemented by the color recommender, according tocertain embodiments;

FIG. 7 illustrates a computer-implemented method to determine color of afood item according to certain embodiments; and

FIG. 8 illustrates a computer-implemented method to providerecommendation to achieve target color attributes for a food itemaccording to certain embodiments.

DETAILED DESCRIPTION

The following description of the embodiments is not intended to limitthe invention to these embodiments, but rather to enable any personskilled in the art to make and use.

In some instances, color of a food item prepared using a set ofingredients may not be as expected, or a chef or another entity maydesire to improve or change the color of the food item without startingfrom scratch. In some other instances, it may be desirable to predictthe color of the food item before it's prepared or cooked so that therecipe can be altered or not cooked at all. Certain embodiments canpredict the color of a food item given a formula for its recipe byutilizing data science, color science, food science and machine learningalgorithms. The formula may include a list of ingredients and theirrespective quantities. Certain embodiments can also recommend changes inthe given formula to achieve a desired color with minimum modificationsto the recipe.

The food item can include plant-based ingredients, animal-basedingredients, synthetic ingredients, and/or a combination thereof. Insome examples, the food item can be a plant-based food item that canmimic animal-based foods from the sensory (e.g., flavor and/or texture)and/or visual perspectives. A number of products (e.g., plant-basedingredients, etc.) are available in the market that can providesubstitutes for animal-based food (e.g., animal-based food such aschicken, meat patties, milk, etc.). For example, a combination ofplant-based ingredients can be cooked using a certain recipe to taste,look and/or feel like sausage.

Certain embodiments may approach the color prediction of the food itemas a regression problem. A color predictor can use machine learningalgorithms to predict the color in a certain color space given a list ofstrings representing the ingredients of the food item. In someinstances, the prediction can be used to determine whether the recipefor the food item needs to be altered or not needed at all even beforethe food item is cooked or prepared. The color predictor canadditionally or alternatively be used as a component in a colorrecommender to provide a recommendation for changes in the formula forthe recipe to improve the color of the food item and/or to achieve atarget color. The color recommender may additionally or alternativelyimplement a loss function to optimize the changes in the given formulaby considering attribute distance, ingredient distance, ingredientlikelihood, ingredient sparsity, and/or other factors. In a specificexample, the color recommender can be based on all of attributedistance, ingredient distance, ingredient likelihood, and ingredientsparsity.

FIG. 1 illustrates a system 100 comprising a color predictor 102configured to predict the color of a food item given its formula (e.g.,based on its formula, etc.). The color predictor 102 may receive aninitial formula 104 of a recipe for the food item and provide colorattributes 106 of the predicted color. The color attributes 106 mayrepresent the color in a color space with floating-point values. Forexample, the color predictor 102 may determine that the color of thefood item is a particular color (e.g., brown, white, yellow, etc.) basedon a color palette or a color scheme.

The initial formula 104 of the recipe for the food item preferablyincludes two or more ingredients and their respective quantities.Additionally or alternatively, the food item may include any suitablenumber of ingredients and/or their respective quantities. The recipe mayinclude plant-based ingredients, animal-based ingredients, water-basedingredients, synthetic ingredients, and/or a combination thereof. Somenon-limiting examples of the plant-based ingredients may includevegetables (e.g., onions, potatoes, peas, garlic, spinach, carrots,celery, squash, etc.), fruit (e.g., apples, pears, grapes, etc.), herbs(e.g., oregano, cilantro, basil, etc.), spices (black peppers, turmeric,red chili peppers, cinnamon, etc.), oils (e.g., corn oil, olive oil,almond oil), nuts (e.g., almonds, walnuts, pistachios, etc.), legumes(e.g., lentils, dried peas, soybeans, pulses, etc.), starch, proteins,fibers, carbohydrates, sugars, and/or other suitable plant-basedingredients, etc. Some non-limiting examples of the animal-basedingredients may include dairy products (e.g., milk, butter, cheese,yogurt, ice cream, etc.), egg-based products (e.g., mayonnaise, saladdressings, etc.), meat products (e.g., burger patties, sausages, hotdogs, bacon, etc.), seafood (e.g., fish, crab, prawns, etc.), and/orother suitable animal-based ingredients, etc. Synthetic ingredients mayinclude artificially produced food, e.g., artificial meats, artificialsweeteners, artificial milk, and/or other suitable syntheticingredients, etc.

As shown in FIG. 1, the initial formula 104 may include a quantity 1(e.g., a first quantity, etc.) for an ingredient a (e.g., a firstingredient, etc.), a quantity 2 (e.g., a second quantity, etc.) for aningredient b (e.g., a second ingredient, etc.), and a quantity n (e.g.,an nth quantity, etc.) for an ingredient m (e.g., an mth ingredient,etc.). The quantities may be represented using percentages, fractions,units, and/or other suitable value types and/or methods. In one example,the initial formula 104 may include 47.2% water, 20.6% pea protein, and15.7% coconut oil, hence the ingredient a (e.g., first ingredient, etc.)is “water”, ingredient b (e.g., second ingredient, etc.) is “peaprotein”, ingredient m (e.g., mth ingredient, etc.) is “coconut oil”,quantity 1 (e.g., first quantity, etc.) is “47.2”, quantity 2 (e.g.,second quantity, etc.) is “20.6” and quantity n (e.g., nth quantity,etc.) is “15.7”.

The color predictor 102 may be configured to perform the colorprediction using a prediction model based on neural networks, regressionmodels, classification models, and/or other suitable machine learningalgorithms.

In certain embodiments, color predictor(s) 102 and/or other suitablemodels, suitable components of embodiments, and/or suitable portions ofembodiments of methods described herein can include, apply, employ,perform, use, be based on, and/or otherwise be associated withartificial intelligence approaches (e.g., machine learning approaches,etc.) including any one or more of: supervised learning (e.g., usinggradient boosting trees, using logistic regression, using backpropagation neural networks, using random forests, decision trees,etc.), unsupervised learning (e.g., using an Apriori algorithm, usingK-means clustering), semi-supervised learning, a deep learning algorithm(e.g., neural networks, a restricted Boltzmann machine, a deep beliefnetwork method, a convolutional neural network method, a recurrentneural network method, stacked auto-encoder method, etc.), reinforcementlearning (e.g., using a Q-learning algorithm, using temporal differencelearning), a regression algorithm (e.g., ordinary least squares,logistic regression, stepwise regression, multivariate adaptiveregression splines, locally estimated scatterplot smoothing, etc.), aninstance-based method (e.g., k-nearest neighbor, learning vectorquantization, self-organizing map, etc.), a regularization method (e.g.,ridge regression, least absolute shrinkage and selection operator,elastic net, etc.), a decision tree learning method (e.g.,classification and regression tree, iterative dichotomiser 3, C4.5,chi-squared automatic interaction detection, decision stump, randomforest, multivariate adaptive regression splines, gradient boostingmachines, etc.), a Bayesian method (e.g., Bayesian Linear Regression,Bayesian Neural Networks, Bayesian Logistic Regression, averagedone-dependence estimators, Bayesian belief network, etc.), a kernelmethod (e.g., a support vector machine, a radial basis function, alinear discriminant analysis, Gaussian processes, etc.), a clusteringmethod (e.g., k-means clustering, expectation maximization, etc.), anassociated rule learning algorithm (e.g., an Apriori algorithm, an Eclatalgorithm, etc.), an artificial neural network model (e.g., a Perceptronmethod, a back-propagation method, a Hopfield network method, aself-organizing map method, a learning vector quantization method,etc.), a dimensionality reduction method (e.g., principal componentanalysis, partial least squares regression, Sammon mapping,multidimensional scaling, projection pursuit, etc.), an ensemble method(e.g., boosting, bootstrapped aggregation, AdaBoost, stackedgeneralization, gradient boosting machine method, random forest method,etc.), suitable classifiers (e.g., naïve Bayes, etc.), and/or anysuitable artificial intelligence approach.

In an embodiment, the ingredients a-m (and/or any suitable number and/ortype of ingredients, etc.) may be represented as a list of strings whichcan be converted to a multi-dimensional embedding vector to representthe ingredients in a meaningful space. As an example, “word2vec”,“Glove”, and/or another suitable ingredients embedding model may be usedto produce the embedding vector. Additionally, a vectorized version of aloss function (e.g., CIEDE2000) may be implemented that can beinterpreted and differentiated by the prediction model. The ingredientsembedding model can be trained by utilizing supervised data,unsupervised data and/or a combination thereof. In some instances, theingredients embedding model may additionally or alternatively be used topredict the probability of each ingredient to be in a recipe.Alternatively, the ingredients can be represented in any suitable form.

The color attributes 106 produced by the color predictor 102 may berepresented as a vector belonging to a certain color space.Alternatively, the color attributes 106 can be represented as anon-vector and/or in any suitable representation. In examples, the colorspace may correspond to CIELAB, RGB (red, green, blue), inverted RBG,HSV (hue, saturation, value), CMYK, and/or another suitable color space.In certain implementations, the color predictor 102 may use the imagesassociated with the food item and identify a most prominent or strongestcolor in the food item for representing in the color attributes 106belonging to the specific color space. In certain embodiments, the colorpredictor 102 is further explained with reference to FIG. 2.

FIG. 2 illustrates a block diagram of the color predictor 102 in certainembodiments.

The color predictor 102 may include an ingredient parser 202, amulti-warm encoder 204, an ingredients embedder 206, an image colorextractor 208, a correction process 210, a color encoder 212, a lossfunction model 214, and a prediction model 216. Additionally oralternatively, the color predictor 102 can include any suitablecombination of the above.

Raw recipe data 218 may be part of a supervised dataset, and/or anunsupervised dataset. The supervised dataset may include input data andoutput data that may be labeled to be used as a training dataset (e.g.,a first training dataset) for future data processing. For example, thesupervised dataset may include data for a recipe for a food item as aninput and a color attribute of the food item as the output. Theunsupervised dataset may include only input data without anycorresponding output data. However, any suitable data types can be usedfor the training dataset.

The raw recipe data 218 may store data associated with multiple recipesof various food items (e.g., one or more recipes of one or more fooditems; in a 1:1 recipe to food item relationship and/or any suitablenumerical relationship between recipes and food items, etc.). In certainexamples, the raw recipe data 218 may store ingredients as a list oftuples of the ingredient name (e.g., string) and its proportion (e.g.,percentage or quota). As an example, the ingredients may include[{“name”: “water”, “quota”: 47.197640117994105}, {“name”: “pea protein(textured)”, “quota”: 20.64896755162242}, {“name”: “coconut oil (withoutflavor)”, “quota”: 15.732546705998034}, . . . ]. The raw recipe data 218may also store recipe as a list of step strings, e.g., [{“step”: 1,“description”: “Weigh 20.65 g of pea protein (textured), 1.57 g of saltand 0.10 g of beetroot powder in a bowl”}, . . . ]. The raw recipe data218 may also store observed colors as a list of strings and one or moreimages of each food item. The raw recipe data 218 may utilize JSONformat and/or another suitable format to store the information. The rawrecipe data 218 may be stored in memory, e.g., RAM, EEPROM, flashmemory, hard disk drives, optical disc drives, solid state memory,and/or any type of memory suitable for data storage.

The ingredient parser 202 may be configured to parse the raw recipe data218. In certain examples, the ingredient parser 202 may process theingredient strings in the raw recipe data 218 to remove any duplicateingredients, remove parentheticals and post-comma descriptors, collapsedouble spaces, remove quantities using list of quantity words, filteroverly-long ingredients and match to clean reference ingredients whenpossible, and/or any other text processing and/or other suitableprocessing. As an example, “organic wild berries (organic blackberries,organic strawberries, organic bilberries, organic raspberries)” may beprocessed to “berries.” Similarly, “½ (12 ounce) box vanilla wafercookies (such as Nilla®), crushed very fine” may be processed to“vanilla wafer cookies.” Processing can be automatic, computational,manual, and/or other suitable types of processing.

The multi-warm encoder 204 may be configured to represent the formula asa sparse proportional vector. In specific examples, the multi-warmencoder 204 functions to generate (e.g., output, etc.) one or moremulti-warm vectors 300, such as based on one or more formulas 302 and/orother suitable inputs. Each dimension of the sparse proportional vectormay correspond to a unique ingredient, and the value in each dimensioncan be the proportion of that ingredient present in the food item. In aspecific example, the proportions are scaled to sum to 1 instead of 100,but any suitable type of scaling can be employed. In a specific example,the multi-warm encoded vector may include entries that are “warm” (e.g.,floating point) instead of “hot” (e.g., one) with most dimensions beingzeros and few dimensions being non-zero. However, the formula can berepresented in any suitable manner.

FIG. 3 shows an example multi-warm vector 300 generated using themulti-warm encoder 204 for a given formula 302. The formula 302 may besimilar to the initial formula 104 in FIG. 1. As an example, the formula302 may include “water: 47.2%”, “pea protein: 20.6%”, and “coconut oil:15.7%” among other ingredients. The multi-warm encoder 204 may encodethe formula 302 to produce a sparse proportional vector comprising [0,0, 0, . . . , 0, 0.472, 0, . . . , 0, 0.206, 0, . . . , 0, 0.157, 0, . .. ] as represented by the multi-warm vector 300. The multi-warm encodingmay be used to perform a weighted sum of embeddings. Since embeddingsare generally trained without quantities, the multi-warm values can bescaled using each ingredient's distribution.

Referring back to FIG. 2, the ingredients embedder 206 may be configuredto convert the ingredient strings into multi-dimensional embeddingvectors that may contain pertinent information about the ingredients.The ingredient embeddings may be generated by training an embeddinglayer using a model similar to an autoencoder for unsupervised featurelearning. In a specific example, the autoencoder is a type of artificialneural network which can be used for dimensionality reduction by copyingthe most relevant aspects of its input to its output. Given a set ofingredient strings, for each ingredient, the embedding layer may providea likelihood of also being in a recipe. In some examples, theingredients embedder 206 may be used to identify ingredients from anunsupervised dataset that may be similar to a first set of ingredientsused in the supervised dataset, e.g., represented by the raw recipe data218 in FIG. 2. Thus, use of the ingredients embedder 206 may bebeneficial to incorporate unsupervised data and domain knowledge tosupplement a limited amount of supervised data. In certain embodiments,this is further explained with reference to FIG. 4.

FIG. 4 illustrates a system 400 which can utilize the ingredientembedder 206 to incorporate unsupervised data (e.g., as a secondtraining dataset, etc.) for training an embedding layer, in certainembodiments.

As shown in FIG. 4, ingredients 402 may include a second set ofingredients belonging to an unsupervised dataset. The ingredientsembedder 206 may be used to convert the ingredient strings from theingredients 402 into multi-dimensional embedding vectors 404 torepresent the ingredients in a meaningful space. In someimplementations, functionality of the ingredients embedder 206 may beadapted from the word2vec (e.g., skip-gram, continuous bag-of-words(CBOW)) and/or similar model, but any suitable approaches canadditionally or alternatively be used. For unsupervised featurelearning, a respective embedding vector for each ingredient in theformula may be fetched from the ingredients embedder 206 to provideembedded vectors 404. An adder 406 may be used to generate an embeddedformula 408 by performing the weighted sum of the embedded vectors 404using their proportions in the formula. Since the embeddings can betrained without quantities in specific examples, the multi-warm vectorvalues can be scaled using each ingredient's distribution. In someimplementations, this may be performed using a dot product with anembedding matrix. The embedded formula vector 408 may have a different(e.g., much smaller) dimension, may be dense, and/or may containnegative values.

The embedding layer 410 may be implemented using a linear neural networklayer and/or another suitable model (e.g., a different type of neuralnetwork, any suitable models described herein, etc.). The embeddinglayer 410 may be trained to incorporate unsupervised data and/or domainknowledge (e.g., a second training set) to supplement the limited amountof supervised data (e.g., a first training set) represented in the rawrecipe data 218. The unsupervised data may belong to a large set ofrecipes obtained from the web, generated automatically or manually,and/or otherwise procured. In specific examples, leveraging theembedding layer 410 (e.g., to incorporate unsupervised data and/ordomain knowledge to supplement supervised data and/or other suitabledata, etc.) can confer improvements in computer function throughfacilitating improvements in accuracy of predicting color of a food itemand/or recommending changes to achieve a target color for a food item.

The activation function 412 may be used to generate a vector 414 with aprobability distribution of the ingredients, which may represent thelikelihood of also being in a given recipe. In some implementations,“Softmax” or a similar activation function may be used to output thevector 414. The probability predictions in the vector 414 can add up toone. As an example, the vector 414 may include “milk: 0.03”, “butter:0.76” and “olive oil: 0.21.” In specific examples, the probabilitypredictions generated by the system 400 may be useful to utilizegenerally large amounts of unsupervised data for supplementing therelatively smaller amount of supervised data for predicting the color(s)of the given food item(s).

Referring back to FIG. 2, the images database 220 may include images inthe form of pixel values (and/or other suitable form) for each food itemassociated with the raw recipe data 218. In some implementations, theimages database 220 and the raw recipe data 218 may be part of the samedataset. In certain embodiments, instead of relying on observed colorinformation of the ingredients or the food item provided in the rawrecipe data 218, color of each food item can be extracted from thecorresponding images. For example, in certain instances, a label for thecolor may be too broad (e.g., brown), multiple colors may be observedfor a food item, and/or the label may include a non-color word. Thus, incertain implementations, the image color extractor 208 in FIG. 2 may beconfigured to extract the color from the image of each food item storedin the images database 220.

The image color extractor 208 may utilize any suitable algorithm toextract the colors from the image(s) of each food item. In someimplementations, given an image, the image color extractor 208 mayflatten its pixels into a list of RGB values. In some implementations,to determine an RGB value for a food item, the image color extractor 208may apply k-means clustering to the pixel values of the food item'simage to get two clusters and then discard one of the clusters which hasthe center closest to the background. For example, the image colorextractor 208 may cluster the color vectors using K-means with k=2 intofood color and background color. In specific examples, the images have auniform white or black background, therefore, in certain instances; fourcorners of the image may be used to extract the most dominant color.Note that the RGB color space is discussed for color representation withreference to the image color extractor 208, however, it will beunderstood that the colors may be represented using any suitable colorspace without deviating from the scope of the disclosure. Additionallyor alternatively, the image color extractor 208 can leverage anysuitable approaches in extract colors from the image(s) of each fooditem.

In certain instances, some of the food items may have multiple separatecolors (e.g., a pie's crust and filling, an empanada's outside color andinside color, a cake and its toppings, and/or food items withirregularly spread out ingredients). In such cases and/or in anysuitable scenarios, preparation process in addition to the rawingredients (and/or in addition to any suitable types of data describedherein, etc.) may be used to accurately predict the color of the image.For example, in some instances, the preparation process can affect thecoloration of homogeneous foods through processes such as baking.

In certain embodiments, the correction process 210 may be used tocorrect any errors for the color extraction process. The images may beinspected manually and/or by automated means for correction. Ininstances, where there are multiple images for a food item, the mostrelevant image may be used. In some instances, raw recipe data for thefood items without any images, or with multiple images with highlyvarying colors may be discarded. Any suitable image processingtechniques can be applied for processing the images into a form suitablefor use by the image color extractor 208. Image processing techniques(e.g., for use in relation to image color extraction, for use inrelation to any suitable components of embodiments of the systems and/orany suitable processes of embodiments of the methods, etc.) can includeany one or more of: image filtering, image transformations, histograms,structural analysis, shape analysis, object tracking, motion analysis,feature detection, object detection, stitching, thresholding, imageadjustments, mirroring, rotating, smoothing, contrast reduction, and/orany other suitable image processing techniques.

The color encoder 212 may be configured to encode the colors of the fooditem for representing in a color space. For example, the colors to beencoded may have been extracted using the image color extractor 208 orprovided in the raw recipe data 218 for the respective food item. Theencoding may be based on RGB space, inverted RGB space, HSV space, CMYKspace, CIE L*a*b* (or CIELAB or Lab) and/or another suitable space. Insome instances, representation may be more or less accurate based on theregression model (and/or other suitable model) used for learning,lighting of the images, and/or other factors. In certain examples,CIELAB may provide better color representation as it is designed aroundhuman perception of color rather than computer display of the image.CIELAB color space is a standard defined by International Commission onIllumination (CIE). It can express color as three values: L* for thelightness from black (0) to white (100), a* from green (−) to red (+),and b* from blue (−) to yellow (+). In some instances, CIELAB colorspace may provide better results for implementing a loss function (e.g.,delta E metric) for the color difference. However, any suitable colorspace can be used.

The loss function 214 may be configured to improve the prediction ofvisually perceived color differences through the introduction of variouscorrections to the color represented in a particular color space. Incertain embodiments, the loss function 214 may implement the CIEDE2000delta E metric to describe the distance between two colors representedin the CIELAB color space. For example, the CIELAB color space may bedesigned so that its Euclidean distance is delta E. The CIEDE2000 deltaE metric may be designed so that its unit distance is the “justnoticeable difference” between the two colors. The “just noticeabledifference” of the delta E can be usually 1, e.g., if two colors havedelta E less than 1 it is unperceivable and larger than 1 it isperceivable. Although, in specific examples, the “just noticeabledifference” is an indirect proxy, it can be a most suitable andobjective metric for human perception and its constants can be tuned tobetter measure the desired color attributes. Furthermore, in specificexamples, CIEDE2000 can be designed to deconstruct the color intoLightness, Chroma and Hue (LCH), and to compensate for nonlinearities inLCH and colors near blue. CIEDE2000 can also include weighting factorsfor L, C, H dependent on the application (e.g., cars, textiles, computergraphics, etc.).

The prediction model 216 may be configured to take into account feedbackfrom the loss function 214 and predict the color attributes 106 of thegiven food item. In certain embodiments, the color attributes 106 may berepresented in the CIELAB space, e.g., L*a*b*. The CIELAB space can berepresented in a three-dimensional space using a three-dimensionalmodel. The three coordinates of CIELAB may represent the lightness ofthe color, its position between red/magenta and green, and its positionbetween yellow and blue. Generally, same amount of numerical changes inthe L*a*b* values correspond to roughly the same amount of visuallyperceived change. It will be noted that the color attributes 106 may berepresented in any suitable color space (e.g., RGB, inverted RGB, HSV,etc.) without deviating from the scope of the disclosure. The predictionmodel 216 may further determine that the color of the food item is aparticular color from a color palette or color scheme.

The prediction model 216 may utilize a regression model, e.g., a type ofneural network and/or ordinary least squares (OLS) and/or other suitablemodels, based on a number of factors. In examples, the effect ofingredients on color is generally not linear, e.g., cooking can changethe color of an ingredient. Furthermore, in non-uniform recipes, certainingredients can dominate the color. Thus, in specific examples, aforward neural network may perform better than the OLS model in certainembodiments. As an example, a configuration of the neural network mayinclude multiple dense layers with leaky ReLU activation and a finaloutput layer with linear activation with embeddings for the input. Boththe input data and the output data may be scaled for mean 0 and variance1.

In certain embodiments, prediction models 216 and/or other suitablemodels, suitable components of embodiments, and/or suitable portions ofembodiments of methods described herein can include, apply, employ,perform, use, be based on, and/or otherwise be associated withartificial intelligence approaches (e.g., machine learning approaches,etc.) including any one or more of: supervised learning (e.g., usinggradient boosting trees, using logistic regression, using backpropagation neural networks, using random forests, decision trees,etc.), unsupervised learning (e.g., using an Apriori algorithm, usingK-means clustering), semi-supervised learning, a deep learning algorithm(e.g., neural networks, a restricted Boltzmann machine, a deep beliefnetwork method, a convolutional neural network method, a recurrentneural network method, stacked auto-encoder method, etc.), reinforcementlearning (e.g., using a Q-learning algorithm, using temporal differencelearning), a regression algorithm (e.g., ordinary least squares,logistic regression, stepwise regression, multivariate adaptiveregression splines, locally estimated scatterplot smoothing, etc.), aninstance-based method (e.g., k-nearest neighbor, learning vectorquantization, self-organizing map, etc.), a regularization method (e.g.,ridge regression, least absolute shrinkage and selection operator,elastic net, etc.), a decision tree learning method (e.g.,classification and regression tree, iterative dichotomiser 3, C4.5,chi-squared automatic interaction detection, decision stump, randomforest, multivariate adaptive regression splines, gradient boostingmachines, etc.), a Bayesian method (e.g., naïve Bayes, averagedone-dependence estimators, Bayesian belief network, etc.), a kernelmethod (e.g., a support vector machine, a radial basis function, alinear discriminant analysis, etc.), a clustering method (e.g., k-meansclustering, expectation maximization, etc.), an associated rule learningalgorithm (e.g., an Apriori algorithm, an Eclat algorithm, etc.), anartificial neural network model (e.g., a Perceptron method, aback-propagation method, a Hopfield network method, a self-organizingmap method, a learning vector quantization method, etc.), adimensionality reduction method (e.g., principal component analysis,partial least squares regression, Sammon mapping, multidimensionalscaling, projection pursuit, etc.), an ensemble method (e.g., boosting,bootstrapped aggregation, AdaBoost, stacked generalization, gradientboosting machine method, random forest method, etc.), and/or anysuitable artificial intelligence approach.

In specific examples, the color predictor 102 may utilize only theingredients of the food item to predict the color since data from theraw recipe data 218 may be limited. For example, the recipe informationcan be unstructured, and there can be insufficient data availablewithout structure. Predicting the color of a food item may not be adeterministic problem due to various factors—multiple dishes withdifferent colors can be made from the same ingredients, many food itemsmay have more than one color (e.g., cakes, pies, soups), limited size ofthe training dataset, and/or other suitable factors. Thus, in such casesand/or in suitable scenarios, predicting a primary color of the fooditem with the recipe information may provide better results. In certainembodiments, the color predictor 102 may additionally or alternativelybe used as a component in a system that may utilize a color recommenderto provide a recommendation for changes in the formula for the recipe toimprove its color to match the desired color. In certain embodiments,this is further discussed with reference to FIG. 5.

FIG. 5 illustrates an embodiment of a system 500 comprising a colorrecommender 502 for providing (e.g., configured to provide, etc.) one ormore recommendations for changes in one or more initial formulas 104 toachieve the desired color of the food item given target color attributes504.

In certain embodiments, the color recommender 502 may be based onstochastic gradient descent (SGD) and back propagation models. The colorrecommender 502 may utilize the color predictor 102 to predict the colorof a food item given the initial formula 104. For example, the colorrecommender 502 can determine one or more recommendations for one ormore changes (e.g., modifications, etc.) to one or more initial formulas104, based on output(s) of the color predictor 102. The colorrecommender 502 may additionally or alternatively implement a lossfunction 510 to optimize the initial formula 104 to achieve the desiredcolor. The color recommender 502 may provide (e.g., output, determine,etc.) a recommended formula 506 and/or new color attributes 508corresponding to the modification in the initial formula 104 to achievethe target color attributes 504. The target color attributes 504 and thenew color attributes 508 may belong to the same color space as the colorattributes 106 of FIG. 1, e.g., CIELAB, RGB, or another color space.Alternatively, the target color attributes 504, the new color attributes508, and the color attributes 106 can belong to different color spaces,and/or any suitable combination of attributes can belong to any suitablecombination of color spaces. As an example, the initial formula 104 mayinclude 47.2% water, 20.6% pea protein, and 15.7% coconut oil; thetarget color attributes 504 may include {R: 102, G: 23, B: 214}.

The loss function 510 may take into consideration multiple factors whiledetermining the changes in the initial formula 104, e.g., attributedistance (e.g., how close is the initial formula 104 to the target colorattributes 504), ingredient distance (e.g., how close is the recommendedformula 506 to the initial formula 104 since it is desired to producethe same or similar type of food item for which the initial formula 104was provided, and in specific examples to have minimal changes to theinitial formula 104, thus making the cooking process similar and/oreasier such as for the chefs, etc.), ingredient likelihood (e.g., howsensible the new formula is, for example, using 80% salt in the recipemay not be typical), and/or ingredient sparsity (e.g., how manyingredients are in the recommended formula 506, for example, it is nottypical to have just a pinch of 300 ingredients). In certainembodiments, the loss function 510 is further explained with referenceto FIG. 6.

FIG. 6 illustrates a block diagram 600 that can be used to describe theloss function model implemented by the color recommender 502 in certainembodiments. The color recommender 502 may be designed to optimizeinputs to the color predictor 102 using the loss function modeldescribed in FIG. 6.

The color recommender 502 may operate on raw parameters 602 for a givenformula of a recipe to implement the loss functions. As an example, theraw parameters 602 may include ingredients in the initial formula 104,e.g., ingredient a, ingredient b, . . . , ingredient m. In someembodiments, the raw parameters 602 may be optimized using a sparsityloss function 604. The raw parameters 602 may be further normalizedusing a normalization layer 606, such as before being operated by otherloss functions, e.g., likelihood loss function 610, attribute distanceloss function 614, and/or ingredient distance loss function 618, etc.,but the normalization layer 606 can be used at any suitable time.

The sparsity loss function 604 may be implemented to bring theingredients in the raw parameters 602 to zero unless they significantlyimprove the attribute loss. The attribute gradient may be balanced withthe sparsity gradient. The slope near zero may be used to determinewhether or not to sparse. The sparsity loss function 604 may alsoimplement a step sparsity loss function, a magnitude sparsity lossfunction, an L1 norm, among others to minimize the color difference.However, the sparsity loss function 604 can implement any suitablecombination of loss functions.

The normalization layer 606 may be used to normalize the raw parameters602 for other loss metrics and to avoid interfering with the sparsityloss function 604. In specific examples, after each training step, rawparameters may be projected according to non-negativity and multi-warmconstraints, and constrained optimization may be performed.

The normalized ingredients may be encoded to generate a multi-warmvector using a multi-warm encoder 608. The multi-warm encoder 608 may besimilar to the multi-warm encoder 204 and the multi-warm vector may besimilar to the multi-warm vector 300 shown in FIG. 3. The colorrecommender 502 may be designed to optimize the ingredients in themulti-warm vector, which can feed to the color predictor 102.

Likelihood loss function 610 may be used to determine whether a givenformula is a real or a typical formula. Likelihood may be defined as theprobability of the given formula under the distribution of formulas,which may be proportional to the product of probability density function(PDF) values for all the ingredients. Likelihood loss function 610 maybe operationalized as log-likelihood for numerical stability. Gradientfor ingredients, which are not in the recipe, can be zeroed out belowtheir minimum. The PDF can be infinitesimal but actual likelihood maynot be.

Likelihood for the original ingredient can be subtracted off for easierinterpretation without any effect on the gradient, as shown in Equation(1) below.

$\begin{matrix}{{{{likelihood}(x)} - {{likelihood}( x_{orig} )}},} & {{Equation}\mspace{14mu} (1)} \\{{wherein},{{{likelihood}(x)} = {\begin{Bmatrix}{f_{X}( x_{i} )} & {x_{i} \geq {minval}_{i}} \\0 & {x_{i} < {minval}_{i}}\end{Bmatrix}.}}} & {{Equation}\mspace{14mu} (2)}\end{matrix}$

Optimizing the likelihood may not always be preferable since it can pushtowards an irrelevant mode. In certain instances, the likelihood can beclipped above a threshold while avoiding any modification of the goodrecipes. In specific examples, the quantities are less likely than theoriginal recipe. In examples, further improvements are possible by usingquantile threshold for all ingredients, only modifying unlikelyquantities similar to the original, or implementing inverse cumulativedistribution function (CDF) instead of log-likelihood, among others.

Ingredient distance loss function 618 may be implemented to determinehow close the recommended formula 506 is to the initial formula 104. Insome examples, it may only include gradient for ingredients that were inthe original recipe. In examples, since new ingredients may change thepercentages of original ingredients, just the selected ingredients maybe normalized. In examples, this may result in those ingredients to be asmaller part of the overall recipe; however, it may be mitigated by thelikelihood component.

In examples, since the ingredient data is not normally distributed andis not linear (e.g., difference between 0.001 and 0.002 is much largerthan between 0.501 and 0.502), logit-normal distribution may be used tocalculate ingredient distance in the logit space. In examples, the logit(inverse sigmoid) function is generally linear near 0.5, can stretchsmall values, and is bounded in [0, 1]. An example logit function isshown in Equation (3), where p is probability:

$\begin{matrix}{{logit}{{(p) = {\log ( \frac{p}{1 - p} )}}.}} & {{Equation}\mspace{14mu} (3)}\end{matrix}$

Logit-normal distribution generally assumes that the logit of the datais normally distributed. The parameters can be μ and σ of the logits, asshown in Equation (4). It may generally be used for modeling variableswhich are proportions bounded in [0, 1], where 0 and 1 may never occur.

${f_{X}( {{x;\mu},\sigma} )} = \frac{1}{\sigma}$

In certain embodiments, recipe score function 616 may be used todetermine L1 distance in nutritional/chemical/physical space. The recipescore function may be based on a difference between the initial formula104 and the recommended formula 504. The score can be determined in anysuitable manner within the scope of the disclosure.

Attribute predictor 612 may be used to determine new color attributes508 corresponding to the recommended formula 506. In someimplementations, the attribute predictor 612 may utilize the colorpredictor 102 or functionality of the color predictor 102 to determinethe new color attributes 508 based on the attribute distance (e.g., howclose is the initial formula 104 to the target color attributes 504).For example, the new color attributes 508 may be similar to the colorattributes 106. The new color attributes 508 may belong to the same ordifferent color space than the target color attributes. As an example,the recommended formula 506 may include 24.5% water, 30.7% pea protein,and 10.2% almond oil, and the new color attributes 508 may include {R:112, G: 15, B: 210}. The new color attributes 508 may correspond to therecommended formula 506, which may provide optimal modification to theinitial formula 104.

Attribute distance loss function 614 may be used to determine how closethe formula is to the target color attributes 504 based on the attributepredictions made by the attribute predictor 612. In some examples, meansquared error (MSE) in the LAB space can be taken into account. This maybe extensible to other attributes (e.g., smell, taste) so long as thereis a differentiable model to predict them given a recipe.

FIG. 7 illustrates an embodiments of a computer-implemented method 700to determine color of a food item according to certain embodiments. Themethod 700 may be executed by the color predictor 102 in FIG. 1 or inFIG. 5, and/or by any suitable combination of components describedherein.

In step 702, a formula for a recipe of the food item may be received.The formula may comprise a list of ingredients and its respectivequantities in the recipe. The food item may include plant-basedingredients, animal-based ingredients, synthetic ingredients, and/orother suitable types of ingredients. As discussed with reference to FIG.2, the formula may be the initial formula 104 received by the colorpredictor 102. The formula may comprise ingredients a-m and itsrespective quantities 1-n. For example, the ingredient a may be waterand the quantity 1 may be 47.2%, the ingredient b may be pea protein andthe quantity 2 may be 20.6%, and so on. In some examples, the formulamay be part of the raw recipe data 218 in FIG. 2.

In step 704, the list of ingredients may be encoded to represent as anembedding vector in a color space. As discussed with reference to FIG.2, the list of ingredients may be encoded by the multi-warm encoder 204or another suitable encoder. In some embodiments, the list ofingredients may be parsed by the ingredient parser 202 before feeding tothe multi-warm encoder 204.

In step 704, color attributes associated with the food item can bepredicted using a color predictor. The color attributes may belong to acolor space. In some implementations, the color predictor 102 may beused to predict the color attributes 106 associated with the food itemas discussed with reference to FIG. 2. As an example, the colorattributes 106 may belong to the CIELAB color space and may includefloating point values.

The color predictor may have been trained using supervised datasetand/or an unsupervised dataset. In one example, the raw recipe data 218and the images extracted from the images database 220 may have been usedto train the prediction model 216 in the color predictor 102. Forexample, the method 700 may also include encoding the list ofingredients to generate a multi-warm vector. The list of ingredients canbe encoded by the multi-warm encoder 204 to generate a multi-warm vectorcomprising a respective dimension for each ingredient in the list ofingredients. In some examples, the list of ingredients may be parsed bythe ingredient parser 202 before feeding to the multi-warm encoder 204.

The method 700 may further include generating the ingredient embeddingvector 404 using the ingredient embedder 206 given the multi-warm vector300. The method 700 may further include extracting color informationassociated with the food item for representing in the color space. Forexample, the color information may be extracted using the image colorextractor 208 and the corresponding colors may be encoded using thecolor encoder 212 for representing in the color space. The method 700may further include training the color predictor given the ingredientembedding vector and the color information. As an example, the colorpredictor 102 may utilize CIEDE2000 delta E metric as a loss functionfor improving the prediction of the color.

In step 706, the color of the food item can be determined based on thecolor attributes. For example, the prediction model 216 may determinethe color of the food item based on the color space the color attributes106 may belong to. For example, the prediction model 216 may determinethat the color of the food item is a particular color from a colorpalette or color scheme.

FIG. 8 illustrates an embodiments of a computer-implemented method 800to provide recommendation to achieve target color attributes for a fooditem according to certain embodiments. The method 800 may be executed bythe color recommender 504 in FIG. 5.

In step 802, target color attributes are received to achieve a targetcolor of a food item given its formula. The target color may correspondto the target color attributes. For example, the target color attributes502 may be received by the color recommender 502. In some examples, thetarget color attributes 502 may be part of the raw parameters 602 inFIG. 6. The target color attributes may belong to a color space, e.g.,CIELAB, RGB, inverted RGB, etc.

In step 804, using a color recommender, changes to the formula can bedetermined to achieve the target color. The color recommender 502 maydetermine the recommended formula 506 as discussed with reference toFIG. 5 and FIG. 6. In certain embodiments, the changes in the formulamay have been determined based on the SGD and back propagation models aswell as one or more loss functions. For example, the loss functions mayinclude the sparsity loss function 604, the likelihood loss function610, the attribute distance loss function 614, and the ingredientdistance loss function 618 to take the ingredient sparsity, ingredientlikelihood, attribute distance, and the ingredient distance respectivelyinto consideration.

In step 806, new color attributes corresponding to the recommendedformula are provided. The color recommender 502 may provide the newcolor attributes 508 corresponding to the recommended formula 506 usingthe attribute predictor 612. The attribute predictor 612 may be similarto the color predictor 102 or include the functionality of the colorpredictor 102, and consider the attribute distance loss function 614 indetermining the new color attributes 508.

As discussed with reference to FIGS. 1-8, the disclosed embodiments canutilize various machine learning algorithms and/or other suitable modelsto help minimize cooking time and resources by identifying even before afood item is cooked or prepared whether a recipe for the food item needsto be altered or does not need to be cooked at all, based on the colorprediction. Additionally, the color recommender can use the colorpredictor to provide a recommendation for changes in the formula for therecipe to improve the color of the food item or to achieve a targetcolor.

Certain embodiments can confer improvements in computer-relatedtechnology (e.g., artificial intelligence, machine learning, neuralnetworks, etc.) by facilitating computer performance of functions notpreviously performable and/or not practically performed by the humanmind. For example, the technology can leverage one or more computationalmachine learning-associated training sets for training one or moreartificial intelligence models (e.g., color predictor models; colorrecommender models; etc.) in a non-generic, application-specific fashionfor predicting color of a food item and/or recommending changes toachieve a target color of a food item.

Certain embodiments can confer improvements in functioning of computingsystems themselves through components (e.g., components of embodimentsof systems described herein, etc.) and/or processes (e.g., processes ofembodiments of methods described herein, etc.) for improvingcomputational accuracy in computationally predicting color of a fooditem and/or computationally recommending changes to achieve a targetcolor of a food item.

Embodiments of the systems and/or methods can include components forand/or processes including generating one or more food items includingone or more colors (e.g., food items with the one or more colors)determined based on recommendations from one or more color recommendermodels and/or color prediction outputs from one or more predictormodels. In a specific example, the method can include determining a setof recommended changes (e.g., to a food item formula) to achieve one ormore colors for a food item; and generating the food items including theone or more colors (and/or including similar colors or derivable colors)based on the set of recommended changes (e.g., implementing one or moreportions of the recommended changes; implementing modifications derivedbased on the recommended changes; etc.).

Embodiments of the methods can include any suitable processes and/orfunctionality described in relation to the figures, system, and/ordescribed herein.

Models described herein can be run or updated: once; at a predeterminedfrequency; every time a certain process is performed; every time atrigger condition is satisfied and/or at any other suitable time andfrequency. Models can be run or updated concurrently with one or moreother models, serially, at varying frequencies, and/or at any othersuitable time. Each model can be validated, verified, reinforced,calibrated, or otherwise updated based on newly received, up-to-datedata; historical data or be updated based on any other suitable data.

Portions of embodiments of methods and/or systems described herein arepreferably performed by a first party but can additionally oralternatively be performed by one or more third parties, users, and/orany suitable entities.

Additionally or alternatively, data described herein can be associatedwith any suitable temporal indicators (e.g., seconds, minutes, hours,days, weeks, time periods, time points, timestamps, etc.) including oneor more: temporal indicators indicating when the data was collected,determined (e.g., output by a model described herein), transmitted,received, and/or otherwise processed; temporal indicators providingcontext to content described by the data; changes in temporal indicators(e.g., data over time; change in data; data patterns; data trends; dataextrapolation and/or other prediction; etc.); and/or any other suitableindicators related to time.

Additionally or alternatively, parameters, metrics, inputs (e.g.,formulas, ingredient attributes, other suitable features, etc.), outputs(e.g., color attributes, recommended formulas, etc.), and/or othersuitable data can be associated with value types including any one ormore of: scores, text values (e.g., ingredient descriptors, etc.),numerical values (e.g., color attributes, etc.), binary values,classifications, confidence levels, identifiers, values along aspectrum, and/or any other suitable types of values. Any suitable typesof data described herein can be used as inputs (e.g., for differentmodels described herein; for components of a system; etc.), generated asoutputs (e.g., of models; of components of a system; etc.), and/ormanipulated in any suitable manner for any suitable components.

Additionally or alternatively, suitable portions of embodiments ofmethods and/or systems described herein can include, apply, employ,perform, use, be based on, and/or otherwise be associated with one ormore processing operations including any one or more of: extractingfeatures, performing pattern recognition on data, fusing data frommultiple sources, combination of values (e.g., averaging values, etc.),compression, conversion (e.g., digital-to-analog conversion,analog-to-digital conversion), performing statistical estimation on data(e.g. ordinary least squares regression, non-negative least squaresregression, principal components analysis, ridge regression, etc.),normalization, updating, ranking, weighting, validating, filtering(e.g., for baseline correction, data cropping, etc.), noise reduction,smoothing, filling (e.g., gap filling), aligning, model fitting,binning, windowing, clipping, transformations, mathematical operations(e.g., derivatives, moving averages, summing, subtracting, multiplying,dividing, etc.), data association, interpolating, extrapolating,clustering, image processing techniques, other signal processingoperations, other image processing operations, visualizing, and/or anyother suitable processing operations.

Embodiments of the system and/or portions of embodiments of the systemcan entirely or partially be executed by, hosted on, communicate with,and/or otherwise include one or more: remote computing systems (e.g.,one or more servers, at least one networked computing system, stateless,stateful; etc.), local computing systems, mobile phone devices, othermobile devices, personal computing devices, tablets, databases,application programming interfaces (APIs) (e.g., for accessing datadescribed herein, etc.) and/or any suitable components. Communication byand/or between any components of the system 100 and/or other suitablecomponents can include wireless communication (e.g., WiFi, Bluetooth,radiofrequency, Zigbee, Z-wave, etc.), wired communication, and/or anyother suitable types of communication.

Components of embodiments of the system can be physically and/orlogically integrated in any manner (e.g., with any suitabledistributions of functionality across the components, such as inrelation to portions of embodiments of methods described.

Embodiments of the methods 700, 800 and/or the system 500 can includeevery combination and permutation of the various system components andthe various method processes, including any variants (e.g., embodiments,variations, examples, specific examples, figures, etc.), where portionsof the methods 700, 800 and/or processes described herein can beperformed asynchronously (e.g., sequentially), concurrently (e.g., inparallel), or in any other suitable order by and/or using one or moreinstances, elements, components of, and/or other aspects of the system100 and/or other entities described herein.

Any of the variants described herein (e.g., embodiments, variations,examples, specific examples, figures, etc.) and/or any portion of thevariants described herein can be additionally or alternatively combined,aggregated, excluded, used, performed serially, performed in parallel,and/or otherwise applied.

The system 500 and/or methods 700, 800 and/or variants thereof can beembodied and/or implemented at least in part as a machine configured toreceive a computer-readable medium storing computer-readableinstructions. The instructions can be executed by computer-executablecomponents that can be integrated with the system. The computer-readablemedium can be stored on any suitable computer-readable media such asRAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), harddrives, floppy drives, or any suitable device. The computer-executablecomponent can be a general or application specific processor, but anysuitable dedicated hardware or hardware/firmware combination device canalternatively or additionally execute the instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the system 500, methods 700, 800, and/or variants withoutdeparting from the scope defined in the following claims.

What is claimed is:
 1. A method comprising: representing, in computerstorage media, an initial formula for a food item as an initial formulavector; representing, in computer storage media, target food colorattributes for the food item as a target food color vector; applying aneural network model to the initial formula vector and the target foodcolor vector to determine a recommended formula for the food item, therecommended formula for the food item comprising changes to the initialformula for the food item to achieve the target food color attributesfor the food item.
 2. The method of claim 1, wherein the initial formulavector comprises a set of ingredients and its respective quantities. 3.The method of claim 1, wherein the neural network model implements atleast one loss function to determine the changes to the initial formula,wherein the at least one loss function considers at least one of aplurality of factors including: an attribute distance between theinitial formula and the target food color, an ingredient distancebetween the initial formula and recommended formula, an ingredientlikelihood of the recommended formula to be a realistic formula, and aningredient sparsity of ingredients in the recommended formula.
 4. Themethod of claim 1, wherein the recommended formula is determined basedon stochastic gradient descent (SGD) and back propagation models.
 5. Themethod of claim 1, wherein applying the neural network model comprisesoptimizing the initial formula vector using a sparsity loss function. 6.The method of claim 1, wherein applying the neural network modelcomprises normalizing the initial formula vector prior being operated onby one or more of at least one loss function.
 7. The method of claim 1,wherein applying the neural network model further comprises: predicting,using a first machine learning model, color attributes associated withthe initial formula, wherein the color attributes associated with theinitial formula are represented as an initial formula color vector;predicting, using a second machine learning model, color attributesassociated with the recommended formula, wherein the color attributesassociated with the recommended formula are represented as a recommendedformula color vector.
 8. The method of claim 7, wherein the initialformula color vector and the recommended formula color vector are usedby an attribute distance loss function to determine an attributedistance between the initial formula and the target food color.
 9. Oneor more non-transitory computer-readable storage media storing one ormore instructions programmed which, when executed by one or morecomputing devices, cause: representing, in computer storage media, aninitial formula for a food item as an initial formula vector;representing, in computer storage media, target food color attributesfor the food item as a target food color vector; applying a neuralnetwork model to the initial formula vector and the target food colorvector to determine a recommended formula for the food item, therecommended formula for the food item comprising changes to the initialformula for the food item to achieve the target food color attributesfor the food item.
 10. The one or more non-transitory computer-readablestorage media of claim 9, wherein the initial formula vector comprises aset of ingredients and its respective quantities.
 11. The one or morenon-transitory computer-readable storage media of claim 9, wherein theneural network model implements at least one loss function to determinethe changes to the initial formula, wherein the at least one lossfunction considers at least one of a plurality of factors including: anattribute distance between the initial formula and the target foodcolor, an ingredient distance between the initial formula andrecommended formula, an ingredient likelihood of the recommended formulato be a realistic formula, and an ingredient sparsity of ingredients inthe recommended formula.
 12. The one or more non-transitorycomputer-readable storage media of claim 9, wherein the recommendedformula is determined based on stochastic gradient descent (SGD) andback propagation models.
 13. The one or more non-transitorycomputer-readable storage media of claim 9, wherein applying the neuralnetwork model comprises optimizing the initial formula vector using asparsity loss function.
 14. The one or more non-transitorycomputer-readable storage media of claim 9, wherein applying the neuralnetwork model comprises normalizing the initial formula vector priorbeing operated on by one or more of at least one loss function.
 15. Theone or more non-transitory computer-readable storage media of claim 9,wherein applying the neural network model further comprises: predicting,using a first machine learning model, color attributes associated withthe initial formula, wherein the color attributes associated with theinitial formula are represented as an initial formula color vector;predicting, using a second machine learning model, color attributesassociated with the recommended formula, wherein the color attributesassociated with the recommended formula are represented as a recommendedformula color vector.
 16. The one or more non-transitorycomputer-readable storage media of claim 15, wherein the initial formulacolor vector and the recommended formula color vector are used by anattribute distance loss function to determine an attribute distancebetween the initial formula and the target food color.
 17. A computingsystem comprising: one or more computer systems comprising one or morehardware processors and storage media; and instructions stored in thestorage media and which, when executed by the computing system, causethe computing system to perform: representing an initial formula for afood item as an initial formula vector; representing target food colorattributes for the food item as a target food color vector; applying aneural network model to the initial formula vector and the target foodcolor vector to determine a recommended formula for the food item, therecommended formula for the food item comprising changes to the initialformula for the food item to achieve the target food color attributesfor the food item.
 18. The computing system of claim 17, wherein theinitial formula vector comprises a set of ingredients and its respectivequantities
 19. The computing system of claim 17, wherein the neuralnetwork model implements at least one loss function to determine thechanges to the initial formula, wherein the at least one loss functionconsiders at least one of a plurality of factors including: an attributedistance between the initial formula and the target food color, aningredient distance between the initial formula and recommended formula,an ingredient likelihood of the recommended formula to be a realisticformula, and an ingredient sparsity of ingredients in the recommendedformula.
 20. The computing system of claim 17, wherein applying theneural network model further comprises: predicting, using a firstmachine learning model, color attributes associated with the initialformula, wherein the color attributes associated with the initialformula are represented as an initial formula color vector; predicting,using a second machine learning model, color attributes associated withthe recommended formula, wherein the color attributes associated withthe recommended formula are represented as a recommended formula colorvector.